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ABSTRACT 


This thesis offers, to those concerned with analysis of modern 
weapons systems, a general methodology for devising appropriate and 
meaningful measures of effectiveness. This methodology does not 
include a specific model for "plugging in'' system parameters and 
mechanically "grinding out" system effectiveness. It is intended 
only as a general "plan" through which the researcher can channel his 
own judgment and experience. The primary purpose of this plan is to 
guide the researcher through a logical transition from a purely 
subjective, and more or less vague, concept of effectiveness to a 
useable and more explicit formulation. 

Effectiveness is modeled as that single system characteristic 
positioned at the apex of a characteristic "pyramid". This pyramid 
is constructed with "layers" of progressively fewer and more subjective 
characteristics. Mathematical properties of measurements appropriate 
to these characteristics are discussed as a function of the intended 
use of the effectiveness measurement. The type of measurement required 
to meet an analysis objective is dependent on the objective function 
or optimization criteria chosen. Because of this fact, the types of 
measurements have been classified into four scales and each scale 


related to a particular type of objective function. 
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des Introduction, 


This thesis is principally concerned with the formulation of 
an expression for the relative or absolute magnitude of a system's 
effectiveness. This "expression of magnitude" is the essence of the 
concept of measurement; but before attempting any plan for obtaining 
such an expression, it has appeared logical to explore some of the 
relevant properties of measurement. The particular manner in which 
this expression of magnitude is accomplished depends on its ultimate 
use in the analysis. In consideration of this fact, it has further 
appeared logical to discuss and catalogue different types of analyses 
and objective functions and relate them to types of measurement. 

The main feature of this thesis is the "characteristic pyramid" 
described in section 7.1. The reader may wish to refer to this section 
initially so as to understand better the direction of the preceding 
sections, or to make a brief appraisal of the model's applicability to 
his own area of interest. The organization of this thesis is directed 
to that reader interested in the general concept of system effective- 
ness. Those seeking a specific model for system effectiveness should 
appraise section 7.1 and utilize other sections as a reference to 


those features of interest, 
1! Justification. 


The validity, and therefore the usefulness, of operations 
research depends upon the skill with which projects are designed, 
and particularly upon the shrewdness with which criteria (such as 


payoff and objective functions) are selected. This criterion problem 


is often relatively neglected in operations research literature, 

and has apparently usually been "solved" in practice by assuming the 
first plausible payoff function that springs to mind; or if several 
spring to mind, by trying all of them and compromising (or letting a 
decision-maker compromise) among the results of alternative computa- 
tions. This problem is much too important for casual treatment. 
Calculating quantitative solutions using the wrong criteria is equiva- 
lent to answering the wrong question. If the methods of operations 
research are applied to the wrong criteria, its quantitative methods 
may prove worse than useless to its clients. 

The terminology, "effectiveness" and "system effectiveness", is 
used extensively by the operations researcher, and is, more often 
than not, included in the objective function. This terminology is 
also applied as a "common denominator" through which to compare alter- 
native systems. A poor choice of a quantitative expression for 
effectiveness is equivalent to selecting the wrong criteria. 

From the multitude of characteristics exhibited by a system, 
there must be selected a set of factors that completely (or at least 
adequately) define the system's effectiveness. Typical characteristics, 
widely used, involve such terms as exchange rates, operability factors, 
probability of kill, and so on. 

All of these parameters used may be further divided into cases 
which are designed to cover range, altitude, speed, and the like. In 
studying any particular system, a particular selection of a system 
of characteristics must be made. The particular selection will depend 
on the particular situation under study. Some characteristics may be 


considered to be of greater importance than others, so that these 


measures may be "weighted" in various ways. It often requires con- 
siderable judgment to select a workable and representative system of 
characteristics to be used as a basic for decision. 

Effectiveness is often conceived as a subjective quantity that 
is not ordinarily amenable to direct, physical measurement. Because 
of this, certain sets of characteristics that can be objectively 
described are chosen to indicate the subjective concept. This is, 
in effect, an extrapolation from the subjective concept to the objec- 
tive characteristics. Unless this extrapolation is logically sound 
and thoroughly understood, it is equivalent to an unintentional, re- 
definition of the system's mission. 

The effectiveness of a system depends on the mission of that 
system. If the mission is narrowly defined, it may be possible to 
measure the system's ability to achieve that mission directly through 
analytical models or experimental testing; however, narrow definition 
of systems and their missions tends to lead to a sub-optimization of 
the system with respect to broader, joint missions. Conversely, 
broader definition reduces the sub-optimization problem among systems, 
but increases the difficulty of obtaining a valid, analytical expression 
for the effectiveness. This suggests that one of the basic considera- 
tions in system definition should be an "optimization" of this measur- 
ability-sub-optimization relationship. 

To be useful as an analytical tool, the concept of effectiveness 
must be defined in such a way so as to permit its characterization 
through a set of physical measurements. The choice and number of the 
elements in this set is important in its own right, but the functional 


relationship between these elements and the concept of effectiveness 


is the most elusive element of the analysis. 

The purpose of this thesis is to focus on the concept of 
effectiveness, to determine the types of measure and methods of 
measurement that are appropriate to various objectives, and to model 
the relationship between the elements of a system and the system's 


effectiveness. 


1.2 Formulating the problem 
Following is a listing of tasks that must be performed, in the 
general order presented, in the evaluation of a system's effectiveness: 
(a) Mission definition. 
(b) System definition. 
(c) Specification of relevant system characteristics. 
(d) Choice of an objective function, 


(e) Construction of a model that is consistent with the 
objective function, 


(f) Data acquisition. 


(g) "Fitting" of the data to the model, 


Hage Measurement, 

Measurement is defined by Peter Caws> as "the assignment of 
particular mathematical characteristics to conceptual entities in 
such a way as to permit (1) unambiguous mathematical description of 
every situation involving the entity, and (2) the arrangement of all 
occurrences of it in a quasi-serial order.'"' The term "quasi-serial 
order" is taken to mean an order that determines, for any two occurren- 


ces, either that they are equivalent with respect to the property in 


lo, W. Churchman and P. Ratoosh, Measurement, Definition and 


Theories (Boston: John Whiley & Sons, Inc., 1962) p. 5 


question or that one is greater than the other, This definition of 
measurement is applied to conceptual entities, which implies that 
before we can hope to measure effectiveness we must have some con- 
ceptual notion of effectiveness. A definition is some statement which 
sets the entity in unambiguous relation with other entities in the 
same or different groups. 

From these two definitions we can see that it may be possible 
to define effectiveness without providing a measurement for it, but it 
is impossible to measure effectiveness without first defining the 
concept. 

The "quasi-serial ordering" requirement can be accomplished in 
four (4) ways which are classified by S,. S. Seevane- into four (4) 
scales of measurement. These four scales are described, along with 
their properties, in Table l. 

This classification narrows our definition of measurement some- 
what, to those relationships between conceptual entities for which 
some property of the real number system can serve as a useful model. 
This restriction is implied if we say that a measurement is the assign- 
ment of numerals to aspects of those entities according to a rule. It 
is the particular property of the real number system, which we choose 
to serve as a model, that determines the properties and applicability 
of the measurement. Some "conceptual entities" can be measured on 
one type of scale and not on another, but the objective of the measure- 
ment may be consistent with only certain types of scales. 


“Teiieepl 25 


qUaMaINnsea_ ay} BuT.IeFFY Woy TM (,x) AeyJoUy Ag peoetTday oq URED eTPDg eB UO (xX) AequMN AUY AION 


sseR 6 yq3uaT dno14 sotzey yo Aqrpenby 
*‘AVISOISAUNA Aqtie{TMITS 243 JO UOT eUTMI9I0g 











ASZ1ouq [eT3UsI0g 
(0964) eanjeradmay, 
(aepuete9) ory, 


SQ0UxIZTJJIGQ FO 20 
STeALaqUL Jo Aqrypenby 
ay3 JO uoT}eEUTWMI9R0q 


uot}zoOUun, But 
-SP2IDUT 91TU03 
-ouow Aue ST 
(X)} ersyA 
(xX)F = ,x 


S[TeBLIUTIW JO ssoupiey 
‘zoqun'] Jo soper1y 
*‘siloquny 332013S 


dno1y SSoJ 10 












2ITU0}OST ‘zaqzee19 SAQt[enby 


JO uoT}eUTUIEI09g 


GoTANnyarTrysqdnas 
2u0-0} 

-ouo Aue ST 
(X)J 21TSqA 
(x)J = a= 


AqtTenby 
JO 


UOT } BEUTMI9799 


s1vkeTd T[[eq oT 
JO ,,suTIoqmNN,, 


*aTeos oy} Wolg 
aT BIS poutmiszeg oq ued 


soil dwexy 


YOTYM 10 eTeIS 24} 


34} jo 


TeotrdAy, 31N}39N1AS 


21n}on14sS a}e015) 07 Aressaoon 


dnoi5 [BOT euUsYyIeW uot}ze1sdg [eotatduy 








INGNGYNS VEN AO STIVOS AO NOLLVOIAISSVIO 


I Wrava 








10 


aig Classification of effectiveness measurement objectives. 


When an attempt is made to find an expression, or measure, of 


a system's effectiveness, some specific use is planned for this 


measure. The nature of this "use" or objective determines the type 


of scale required to meet the objective. 


2.1.1. Comparison of several alternative systems or configurations 
to determine the least cost for a given effectiveness, i.e., 
Minimize Cost 
so that 
Effectiveness = Constant 
Since there is no requirement for a knowledge of the absolute or 
relative magnitude of the effectiveness, the most general scale that 


will satisfy the objective is the nominal. 


2.1.2 Comparison of a discrete set of alternative systems or configu- 
rations to determine the greatest effectiveness for a cost less than 


or equal to some given amount, or conversely, i.e., 


Maximize Effectiveness Minimize Cost 
so that or so that 
Cost ee Constant Effectiveness s Constant 


There is no requirement for a knowledge of the absolute magnitude of 
the effectiveness, but we must be able to order a set of discrete 
alternatives. The most general scale that will satisfy this objective 


is the ordinal. 


2.1.3 Comparison of a "continum"” of system configurations to 


determine the greatest effectiveness for a cost less than or equal 


1 


to some constant, or conversely, i.e., 


Maximize Effectiveness Minimize Cost 
so that or so that 
= = 
Cost <Q Constant Effectiveness >> Constant 


This objective is essentially the same as 2.1.2, but we are required 
to maximize a continuous function as opposed to choosing a maximum 


from a set of discrete quantities, 


2.1.4 An objective that falls naturally into this class is the 
determination of the rate of increase (or decrease) in effectiveness 
as a function of cost. There is no requirement for a knowledge of 
the absolute magnitude of the effectiveness, but we must be able to 
compare amounts of increase in effectiveness as a function of the 
costs of those increases, The most general scale that will satisfy 


this objective is the interval scale. 


2.1.5 Determination of the absolute effectiveness of a system, so 
as to make utility comparisons of cost-effectiveness combinations. 
Compare: xX Effectiveness and Y Cost 
with X' Effectiveness and Y' Cost, 


where X >>X and yY >yY ,» and make the "best'’ choice, 


This objective requires a measurement that provides a clear concept 
of the absolute magnitude of the effectiveness, and can be provided 


only by the ratio scale, 


3. Cost, 


All of the objectives discussed and classified above concern 


some type of cost-effectiveness trade-off. The term "cost" is not 


iz 


restricted to the narrow definition of "dollar cost", but includes 

the general idea of resource requirements. Almost all system analysis 
is performed using the basic concept of some sort of trade-off between 
resource and effectiveness. To accomplish this trade-off, these two 
system parameters (cost and effectiveness) are separated, sometimes 
rather artificially, and envisioned as separate entities. This 
distinction is not, however, theoretically necessary. We could define 
the mission so as to include resource requirements and define the 
single resultant entity to be the system effectiveness. This concept 
of a system's effectiveness would then imply only a single, unambiguous 
criterion for choice, namely, maximize effectiveness. This is not, 
however, the approach taken in this thesis, and the usual distinction 


between cost and effectiveness will be retained. 


4, A hypothetical example of the scales of measurement appropriate 


to various objectives. 


System definition: The warhead to be utilized on an existing 
missile, 
Mission: To produce a specified “overpressure 


distribution" over a circular area centered 
on the detonation point. 

Suppose that, originally, the only information available con- 
cerning the blast effects of various alternatives is that some produce 
identical distributions and others produce different distributions. 
From this primitive information we can place the effectiveness of the 
warheads on a nominal scale. If we establish a criterion of choosing 
the warhead with the least cost subject to its having an effectiveness 


equal to that of some arbitrarily selected warhead, we can use this 


SS 


scale for selection of the best warhead, 

Suppose that through testing and/or analytical methods we now 
learn that the values of this distribution are related to some measur- 
able parameter of the warhead through an "increasing, monotonic 
function", If this is the only information available concerning this 
function, we can place the effectiveness of different designs on an 
ordinal scale. If we establish a criterion of choosing the warhead 
having the greatest effectiveness for a cost less than or equal to a 
set amount (or conversely), we can pick the best warhead. 

Through further testing or analytical investigation we may learn 
the functional relationship existing between the design parameter and 
the effectiveness to within a multiplicative factor and an additive 
constant. 

This would be expressed as: 
Effectiveness = U.f (design parameter) + b, 


where f is known, and U and b are unknown constants. 


This scale is the most widely used for effectiveness because it requires 
no knowledge of the absolute magnitude of the effectiveness and the 
choice of units (U) is completely arbitrary. Potential energy offers 

an example of a physical quantity measured on this scale. 

If we are given the potential energy of object A as 10 "units" 
we must be furnished additional information concerning the reference 
point and the nature of the units before the statement is useful in 
itself. If, in addition, we are given the potential energy of object 
B as 20 "units", and told that the reference points and units of the 
two measurements are equal, we can make meaningful comparisons between 


the potential energy of the two objects, 
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Returning to our warhead example, we can make a direct measure- 
ment of the tons of explosives contained in the warhead. Assuming 
that f is the cube-root function, we construct an expression for the 


effectiveness of the warhead in the form: 


/3 


Effectiveness = Uf (tons of expreeime) * aly" 


If design A has effectiveness = U.f(x) +b, and cost Y dollars 


design B has effectiveness = U-£(x') + b, and cost Z dollars 


design C has effectiveness = U.f(x'') + b, and cost W dollars, 


we can make a statement of the form: 


For (Z-Y) dollars we can increase the effectiveness by U+£(x')-U- £(x) 
above the effectiveness of design A, and, 


For (W-Y) dollars we can increase the effectiveness by 


(U-£(x |) - U-£(x)) 


above that of design A. 


This statement provides the rate of increase of effectiveness as a 

function of the cost of that increase, and we may be able to make a 

choice between designs A, B, and C on the basic of this information. 
Finally, we may be able to obtain the exact value of the over- 

pressure as some function of design parameters. Since the mission of 

the warhead was defined "to produce a specified overpressure", we 

can obtain the overpressure of a particular design, compare it to the 

specified overpressure, and make a statement of the form: The effective- 

ness of the design is XZ (or warhead A is X% effective). An extremely 

important point to bear in mind, in connection with this example and 


in an actual analysis, is that we are expressing the ability of the 


15 


warhead to achieve its expressed mission and nothing else. If the 
mission of the warhead were intended as the accomplishment of certain 
damage to the target, there is no guarantee that the ability to achieve 
X% of the overpressure specified will, in fact, produce X% of the 

damage which would have resulted from the specified overpressure. 

This is a measurement that can be placed on the ratio scale and requires 
that the units of the quantity measured be identical (or differ by a 
known multiplicative factor) with the units expressing the mission 
attainment. In addition, it requires a common reference point which, 


in our example, is assumed to be the following: 


Zero overpressure corresponds to zero effectiveness. 


Having a measurement of effectiveness on this scale permits a 
criterion of "maximum utility" for alternative cost-effectiveness 


combinations, 


om Units for Effectiveness. 


Karl Meneer has shown that the variables of physics such as 
work, heat, energy, acelleration, etc., cannot be regarded as the 
class of numbers; nor can they be regarded as the class of physical 
entities. They must be regarded as the class of pairs (n, E) such 
that one element of the pair is a real number and the other is an ele- 
ment from the class of physical entities. 

Effectiveness can be considered as a physical entity. In fact, 
it is a characteristic of a physical system in this discussion. It 


cannot be expressed as only a number any more than it can be expressed 


3 

Menger, Karl, On Variables in Mathematics and Natural 
Sciences, British Journal for the Philosophy of Science, Vol. no. 18, 
1954, p. 135. 


16 


as only a physical entity, if the expression is to be heed in 
quantitative analysis. 

Three commonly used expressions of effectiveness may appear 
as contradictions to this idea. These are, (1) Probability, (2) 
Reliability, and (3) Ratios. All of these expressions imply the "unit" 
of complete or total effectiveness and they must include a lucid 
description of this "unit” to be meaningful. For a proposed measure 
of effectiveness to be placed on the ratio scale and expressed in 
units of U (say p.s.i), the mission of the system must be expressable 
in U and, in addition, the number of these units U necessary to 
achieve the mission must be known. 

Probability is commonly understood to represent the frequency 
of some specified event in relation to the number of opportunities 
for this event to occur. The statement that the probability of obtain- 
ing "heads'' on any one toss of a coin is 1/2 means that the frequency 
with which this event will occur (over a long series of trials) is 
1/2 of the opportunities afforded, If probability is used as a measure 
of effectiveness we are saying that the system will exhibit complete 
effectiveness with the stated frequency. It is necessary to under- 
stand exactly how this "complete effectiveness" event is characterized 
before the probability statement can assume any meaning. For example, 
suppose "probability of kill" is used as the measure of effectiveness 
for some system. The implicit assumption is that each time the system 
exhibits the "kill event"’ it is completely effective for that trial, 
and further, if it exhibits the kill event each and every time over a 
long series of trials, it is completely effective (can accomplish its 


mission with certainty). The unit of effectiveness is the "kill event” 
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and the number of these units necessary for complete effectiveness 
must equal the number of trials. 

Reliability is just that terminology applied to the probability 
of accomplishing a mission, and has the same interpretation as a 
probability statement. 

A ratio can be used in the same manner as probability, except 
that it is usually an "a posteriori" statement rather than an "a priori" 
statement. A ratio can also be interpreted as the magnitude of some 
system characteristic relative to some standard magnitude. Used as a 
measure of effectiveness, this implies that the "standard magnitude" 


of the characteristic will endow complete effectiveness, 


On Measurability and Sub-Optimization. 


The usual task of quantitative analysis is the improvement of 
decisions at relatively low levels (efficiency ‘in the small"). 
Optimum decisions at low levels do not, however, imply an optimal 
solution to higher level structural decisions. If the Navy attempts 
to optimize the design of an aircraft carrier, taking the incluston of 
a nuclear power plant as given, then it runs the risk of sub-optimiza- 
tion within a frame work in which the type of power plant is not 
assumed, but is a variable of the analysis. Suppose we have three 
sub-systems, A, B, and C, which are components of a larger system S, 
The effectiveness of system S is a function of the effectiveness of 
the sub-systems. Each of the sub-systems has a mission which, in 
combination, contribute to the mission of S. Optimizing the design 
of the sub-systems, according to some criterion that seems appropriate 


to their individual missions will not, in general, optimize the 
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criterion appropriate to the composite system S. 

Because of the sub-optimization problem, there has been a marked 
effort within the Defense Department to define systems in a broader, 
more inclusive manner. If the optimization process is applied correctly 
to the broader systems, sub-optimization among components is (in 
theory) avoided. It might seem that this problem could be entirely 
eliminated by defining the "system" as the complete defense establish- 
ment, and performing the optimization directly on this "system". 

There are any number of obvious reasons why this would be an impossible 
task; but there is one, somewhat more subtle, difficulty that is not 
immediately obvious. This difficulty tends to dominate the process 
even at relatively low levels, and has to do with expressing the 


effectiveness element of the optimization criterion. 


6.1 Optimization. 

Any optimization process implies an operation on two or more 
characteristics of the system. If we optimize the mix of sand and 
cement to obtain the strongest concrete, the two elements are the "mix" 
and the strength of the concrete. A further requirement is that some 
functional relationship exist between these elements. Here, the 
strength of the concrete is a function of its "mix". Alternatively, 
we can optimize the strength of the concrete subject to some cost 
constraint. If both the cost and strength of the concrete are functions 
of the mix, then this "mix" can be the parameter which relates cost 
to strength. In military systems analysis the two elements of the 
optimization process are generally the mix of elements (or design) 
and the effectiveness; or else the cost and effectiveness. It would 


be difficult to imagine an optimization function that did not include 


iy 


the concept of effectiveness (although the terminology might be 
different). The concept of effectiveness contains the purpose of 
conceiving or building the system, and optimization implies an opera- 
tion concerned with the purpose of the system. The first step in 
planning an optimization criterion is the selection of the elements 


that the process is to operate on. 


6.2 Measurability 


As the system becomes more broadly defined, envisioning the 
mission as a unique entity becomes more difficult, It may be necessary 
to state the mission in broad subjective terms, or even as a set of 
"sub", or alternative, missions, Examples of broadly defined missions 


are ; 


(1) Provide a deterrent against enemy aggression, 
(2) Provide a retaliatory capability, 


(3) Provide a strategic nuclear delivery capability and 
an aerospace research vehicle, 


These missions suggest no easily conceived, unambiguous, entity which 
can be used as the second element of the measurement set. To obtain 
this element, and a measurement that can be included in the optimiza- 
tion process, some characteristic of the system is chosen that con- 
tributes to the achievement of its mission. Unfortunately, the 
effectiveness of the system usually derives from additional character- 
istics in combination with the one measured, The difficulty of 
combining this set of characteristics into a single element that can 
be logically included in the optimization process increases as the 


system becomes more broadly defined, 
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6.3. Sub-Optimization of Characteristics in Broadly Defined Systems 


When the effectiveness of a system is represented by a measure- 
ment of a contributing characteristic, rather than by a measurement 
on the set of all contributing characteristics, and this measurement 
is included in the optimization process, we are actually sub-optimizing 
between characteristics just as we sub-optimized between components in 


narrowly defined systems. 


Vs System Model. 


A system can be modeled in the following way to indicate the 
manner in which effectiveness is derived from the basic resources 
employed in the system. A hypothetical missile system is used to 


provide representative examples for the notational elements. 


Let X12 Xo Xg> » X, » represent the basic resources used in 
the system. These can be single, 
initial, requirements or a flow per 
unit time. 


For example: 


x, = tons of steel plate 

X= gallons of propellent 

X, = number of personnel 

x, = megatons of explosives 

Xe = number of electronic components 
x. = etc. 


Zi 


Let a, ae i= 1,2,3, °°*°, 8s, j =1,2,3, °°°*°, © represent the 
3 


ads production process operating on the fs resource, 


Let a be the matrix of the a 7s 
Bale) 1,4 


For example: 


ai 4 cut the steel plate to dimension D. 

3 

aro = form the steel plate into cylinder C. 
3 


a, 3 = apply red paint to the steel plate. 
3 


= etc, 


Now if the X98, i#1, 2, 3 °**’,r are considered to constitute a 


row vector X , and we form the combination’; 


X (ay 4) 


then Y is a column vector having components; 


@eee- 


Y4? Yo9 ¥32 cle 
where y, 184 basic component (or unit) of the system. 


For example: 


yy missile airframe 


Yo guidance system 


4 ee 
The elements of X are related to those of (a, ? just as in 
3 


ordinary vector multiplication, but these elements are not combined 
through ordinary multiplication. The element of (a, Ry "operates on!’ 


its corresponding element in X to form anelement of Y. The nature of 
this operation will be evident in context. 
22 


cn propulsion system 


/ a warhead 


y eee. 


Let Dey ao = L233 * ese v = 1,2,3, «+++, t represent the 


3 


th : j — 
V tactic, employment, or environmental condition 


; th ; 
operating on the wu basic component. 


and (b__) be the matrix of the b 4B 
u,v u,v 


3 


For example: 


by i, ee two warheads on each airframe. 
3 
by i; protect each missile with a silo. 
3 
by 50 representative characteristic of enemy antimissile 
* defense. 
=e 
Paik EGe. 


Now since Y is a column vector, form the combination; 
v (b = ¢ 
gy? C 
Then C is a column vector having components; 


Cc Ce eeeoes c 


yor te 


th , ona 
where c,, is the h "basic" system characteristic. 


Ze 


For example: 


Cc, = probability that a missile reaches the antimissile 
defense area, given that it is launched. 
Cc, = range of early warning radars 
C2 = reliability of guidance system 
Cc, = warheads damage radius. 
a = etc. 
t 


It is assumed that the cS are such that adequate, quantitative 


expression can be obtained for them. 


7.1 System Characteristics. 


In relating the Ch ?Ss h = 1,2,3, °°°*»s t, to a single expression 
for effectiveness, it is helpful to utilize set notation. 
Let the set of all egies h = 1,2,3 °°*°°,t, be denoted by of) 


where the superscript refers to the "level" of the elements. Upon 


adding the superscript to the elements we have: 


(M2 M.D, Dw, 
(1) 


Now there will exist e sub-sets of C where e<t that have 


the property of completely describing the effectiveness of the 


system, if suitable functions are defined on these sets. 


1 e eee 
Let s\ ye a ee A » e@ , denote the ha such sub-set of c(h), 
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Let c 62) = ae, ey a) ieee a represent the set 


of''second level'"' characteristics whose elements are functions of the 


gl ee 
J 


This "second level'’ set of characteristics represents a manner 

of describing the system's effectiveness through a smaller, and more 
general, set of parameters, The lowest, or first, level of character- 
istics are all those numerous, and distinct attributes of a system 
contributing to its effectiveness. The elements of the "second level" 
set are combinations of these basic attributes that describe a slightly 
more general system characteristic. For example; suppose two basic 

(or first level) characteristics of a missile system are speed, and 
burn time. One element of the "second level" set would be the range 

of the missile as a function of these two basic characteristics. 
o(2) 
J 


Cite. = (speed) (burn time)) 


Then: 


2 1 ; ae es 
o! ys f. (85 ) EE heeds: 2 TOR > e , where 


si 1 is the function that relates the Py set of first 
3 


level characteristics to the ED second level character- 
rsere. 
Reviewing the notational scheme we have: 


- = the 7s k-level, characteristic of a system. 


c6*) = the set of k-level characteristics completely describ- 


ing the system's effectiveness. 
a = the ba sub-set of c(h) | 
By aan the function relating the a sub-set of k-level 
J» characteristics to the jth (k + 1)-level characteristic. 


ZS 


Then: 


(k+l). (k) = is 
C4 Te yr. 2 es Diss = 


where e is the general "termination" of the index j. 


Now eis a "strictly decreasing" step function of k, i.e., as 
we express the effectiveness of a system in terms of higher level 
characteristics, the number of these characteristics necessary to 
describe fully the effectiveness decreases. Finally, there will 
exist some k for which e# 1. This single characteristic, then, 


fully describes the effectiveness of the system and, if suitable 


(1), 
j 


of the effectiveness. These restrictions will be discussed later, 


restrictions are imposed on the s- x’8 and c s, is a measurement 
3 
In common with many notational schemes or models, the notation 
here is much more difficult than the idea that the model is intended 
to convey. The following diagrammatic representation of the mathe- 


matical model is presented to augment the notation. (Figure 1) 


7.2 Properties of the acing and f 98-6 
j j,k 
The properties that must be possessed by the ons and 
a . 8 depend on the objective of the effectiveness measurement, 
3 


(see section 2.1) These properties are listed below in connection 


with analysis objectives, 


(1). If the measure of effectiveness is required only for the 


determination of equality; the of) 


j 


measurement suitable for the nominal scale, and the set of ay k 
9 


> je 2,3,°°**, t ean be eny 
°s are 


required only to provide a one-to-one substitution, 
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(2) If the measure of effectiveness is required only for an 


(1), 
j 


ordering" among alternatives; the c 8 can be any measurement 


suitable for the ordinal scale, and the f *s can be any monotonic 


j,k 
increasing functions. 
(3) If the measure of effectiveness is required for the 


(1), 


determination of marginal rates of increase; the a s can be any 
measurement suitable for the interval scale, and the ne can be 
any linear functions whose constants need not be known. 

(4) If the measure of effectiveness is required as one of the 
elements in a cost-effectiveness combination that is to be compared 
for maximum utility; the AAD must be "absolute" measures on a 


ratio scale, and the f, ,’s must be linear with all constants known. 


j,k 
This linearity does not refer to the individual variables, but rather 


to the set of variables over which the function is defined, 


8. Measurability of the eu ag 


We assume that the basic characteristics of the system, which 


(1) 


are denoted c » are easily conceived, conceptual entities. Now 

there are two alternative methods of obtaining the next highest level 
set of characteristics: (1) direct physical measurement through some 
type of experimental testing, and (2) analytical modelling of the set 


of functions f j = 1,2,3,°°°*, e . Both of these tasks become 


j,i ’ 
more difficult as the level (k) of the characteristics increases. 

Because of this increase in difficulty, some level (k) is reached 

such that it is either impossible, or we are unwilling, to express 
the (k+1l)-level of characteristics with any degree of certainty. 


Since the termination of the index 4 (e) is a function of k, there 


will exist e (k) k-level characteristics that contain all information 
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contributing to the effectiveness of the system. We are now presented 
with the problem of finding some function E, defined over the e(k) 
elements of ok) | that will provide a single, quantitative expression 
for the effectiveness of the system. The difficulty of finding or 
formulating this function will increase as the number of variables 
(e(k)) increases. This "trade-off" between measurability of individual 
system characteristics, and difficulty of obtaining the effectiveness 
function defined over this set (of characteristics) is encountered 
in every system analysis. The ability to resolve successfully this 
trade-off will also depend on the generality of the system definition 
and mission; and this in turn, affects the sub-optimization problem 
between systems. This interdependence of concepts is indicated in 
Figure ITI. 

From the preceding discussion (and Figure II), it can be seen 
that an important consideration in the system and mission definition 
phase should be a satisfactory "trade-off" between the sub-optimization 


among systems, and that among the characteristics of these systems. 


9. The Effectiveness Function. 


Let E represent the effectiveness of a system. 


Then: 


where: 


a3 


= eC oe he —* ap 


b 


k is the highest level of measurable characteristics and 
e is the number of these characteristics necessary to 


completely describe the effectiveness. 
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M. C, Heuston and G, Opens? have listed 5 properties that this function 
must possess, and that will provide an ordering relationship comparable 
to the ordinal scale. 

(1) Its domain of definition is that part of euclidean e-space 
satisfying the condition that a point is in that set when all its 


(k), 


components (i.e., the ves s) are all non-negative. This non-negative 
requirement can always be satisfied through a simple translation of 
the origin. 

(2) Its range (i.e., the value of the function) is also non- 
negative. 

(3) When it is set equal to some constant, the resulting 
contour will define a hypersurface. 

(4) Its first partial derivatives are all positive. This means 
that as any one characteristic is increased (all others remaining 
constant) the effectiveness of the system will increase. 

(5) It is strictly quasi-concave- a property that can be 
shown to be equivalent to the "law of diminishing marginal returns". 
This means that as any one characteristic is increased (all others 


remaining constant) the effectiveness will increase, but at a decreasing 


rate. 


These properties may provide some insight into the nature of this 
function, but are of little help in its construction. This task 
is usually (and necessarily) left to a decision maker who constructs 


the function in the form of his personal utility function. Assuming 


“Hewson . M. C. and Ogawa, G., Observations on the Theoretical 
Basis of Cost-Effectiveness, Journal of the Operations Research Society 
of America, Vol. 14, No. 2, p. 242. 
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that the decision-maker's objectives are consistent with national 
objectives, the analyst's only concern is that the number and nature 
of characteristics presented to the decision maker are such that a 
well-formed, consistent utility function can be defined over this 
set (of characteristics). Whether E is considered to be some 
mathematical relationship or a utility function, the difficulty in 


(k) 


obtaining a "good'"' E is dependent on the number of elements in C 


9.1 A Mathematical Model of E., 


A common model that is found in a wide range of "effectiveness 
literature" possesses all but number 5 (quasi-concave) of the proper- 
ties proposed above. This model provides a framework on which to 


apply relatively simple utility functions of only one variable. 


f=e 
E= [we ce™ | (equation #2 
f=] 
where the w,’s are arbitrarily assigned "weighting factors" 


that represent the utility of the individual een 7a. 


This model has the appeal of simplicity, but possesses serious short- 
comings when applied over a wide range of characteristic values. 

Its primary application is envisioned as "ordering" the effectiveness 
of several systems possessing identical types of characteristics. 
Unless this expression is envisioned as absolute, no suggestion is 


given concerning the comparison of.systems not possessing identical 


type characteristics. 
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10. Conclusion. 

From this analysis of the concept of system effectiveness, the 
following list of tasks, considerations, and decisions should be 
followed in progressing from system definition to a quantitative 


criterion of its value. 


1. The mission should be defined as broadly as possible, 
consistent with some concept of how its ability to achieve this 


mission can be expressed quantitatively. 


2. The system designed or envisioned to accomplish this 
mission should be explicitly defined out to some "boundary". This 
boundary must separate the system from its environment, and contribu- 
tions to this mission from other elements or systems (outside this 


boundary) is incidental and outside of their own missions. 


3. A criterion for judging the value of the system, or for 
choosing between alternative systems or designs must be formulated, 


and/or, 


4, a method of optimizing the design or choice of the system 


must be devised. 


5. Based on the optimization method chosen, certain types of 
measurements must be obtained for a complete set of characteristics 


at the highest level possible. 


6. Depending on the number of elements in this set, a method 
of expressing the effectiveness of the system as a function of these 


elements must be designed, or if this cannot be obtained with the 
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desired confidence of its correctness, 


7. the mission must be re-defined in such a manner that the 


effectiveness can be more confidently expressed, 
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APPENDIX I 


A Simplified, hypothetical missile system, 


Description: 


Mission: 


Resources; 


100 ballistic missiles, silo protected, with associated 
early-warning radar system and command and control 
system, 


To deter the enemy from a premptive strike, 


= steel plate 

= propellent 

= explosives 

= electronic components 
= concrete 

= labor 


= operating personnel 


Production processes: 


oi 


Soh 


= form steel plate into cylinder C. 


= mix propellent into formula F 


= assemble guidance systems G 


= 0 


= assembly processes 


36 


Basic system element: 
(=7 
Y, = ) (ia) = missile airframe 
l (complete except for 
aoe warhead) 


Production processes: 


aio = form shape B 

ao = 0 

Azo = shape explosives into shape S$ 
ang = assemble fusing mechanism 
any = 0 

ago = assembly 

aay = 0 


Basic system element: 


L=7 

Yo = | (X:di 2) = warhead assembly 
=] 

Production processes: 

a43 = form radar reflector R 

oo) 

235) 

a13 = assemble radar transceiver 

Ax. = build radar sites 

aga = assembly and construction 

As. = staff with operating crew 
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Basic system element: 


(=7 
Nig (X:Qj 3) = radar early-warning system 
4 
i=] 
Production processes: 
a 92, 4. 23qe Oona 


ayy, = assemble communication equipment 
ag, = assembly and construction 


as, = staff command & control system with operating crew 


Basic system element: 


é=7 

Viele » Ais) = command and control sub- 
é system 
Ze / 

Production processes: 

aig = form silo hardware 

ans * 0) 

azo = 0 

eis) 4. © 

aco = build silos 

acs assembly and construction 

ase = staff silo with launch crews 


Basic system element: 


L=7 
= Y Widys) = at 
isf 
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Then: 


(Xy 5X5 2X4 5%y 2X5 oX—oX7)] | 2), 470 2445 


"ll 9475 


Tactics and environmental matrix. 
cp Sw 


airframe 
warhead 
early-warning 
system 


command & control 


silos 


Enumerating the elements of (b 2) is not enlightning because 
3 


of the generality necessary. This matrix imposes tactics, procedures, 


and environmental conditions on the system elements producing the 


basic system characteristics. 


Then: v B = a burn time of propulsion system. 
ofl) flight fi | 
g profile parameters. 
ob = accurac i id 
3 y of guidance system. 
oe = probability of penetrating anti- 
missile defense. 
6!) = reliability of airframe. 
a) = reliabilit f id 
6 y of guidance system. 
AO = ran 1 1 i d 
7 ge o early warning radar. 
cu = reaction time of silo launch crew. 
ee = reaction time of command and control 


system. 


ay, 


ah 
“10 


(1) 
mall 


oo) 
=e 


(1) 
“ie 


Second level set of characterist 


Sub-sets of the set of) 


heigher level characteristics. 


CL) = (1) (1) .(1) 
Ee WED Ca Ge 


OY (1) 
Then: Cc, = 2h 183 ) 


(1) _ CL eeCl) -SECE) 
eee ce 


Ae) (1) 
Then: C4 f, 1685 ) 


s) - of, (D, (0) 


» Cg 
Then: cy” = f, “esr 


oS) 


4, 10? aslaliaatatse 
Then: ae = Ey ee ) 
fo CD 


Then: a = Bier ) 


reliability of fusing mechanism, 


fusing height. 


overpressure distribution of warhead. 


blast resistence of silo. 


ics. 


are now chosen to represent a set of 


) = (burn time, flight profile, 


accuracy of guidance system) 


C.E.P of missile at specified 
targets 


) = (probability of penetration, 
airframe reliability, guidance 
system reliability) 


probability of missile reaching 
target. 


) = (radar range, silo launch time, 
c&c launch time.) 


probability that missile can be 
launched if attacked. 


(fusing reliability, fusing 
height, overpressure) 


damage radius centered on 
explosion point. 


= (blast resistance of silo.) 


= damage radius necessary for 
attacking missile. 
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(2) 


All the elements in C can be obtained, either through experimental 


testing or by obtaining + analytically. 
3 


1 
Third level set of characteristics, 


Sub-sets of ¢ 64) can now be chosen to represent the set, 


(c6) of third level characteristics. 


a = ( ae on ) = (C.E.P. damage radius ) 


Then: a” = fy aan ) (expected damage at specified 
b 


targets.) 


(probability of missile reaching 
target, probability that missile 
can be launched, damage radius 
necessary for attacking missile) 


Heerce’?), yc?) - 


Then: or = f, Oe a ) probability that missile will 


reach target area. 


(3) 


These elements of C cannot be obtained through experimental testing, 


but the functions (f f., >) can be constructed with reasonable 
bd 


Poe 


confidence, 


Effectiveness function. 


We now have a set of characteristics containing only two, well 
defined, elements which completely describes the effectiveness of the 


system. 


EB. -=se brit oe a ) 


If we believe, as has been assumed here, that the ability of this 
missile system to deter the enemy from conducting a premptive strike 


is a function of these two elements only, any information concerning 
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E will contribute to obtaining a measurement for the effectiveness. 


(3) (3) 
2 


Since Cy and c are both measurements on a ratio scale, it is 


possible to express the effectiveness of the system in absolute terms. 
a 


i oe < a and/or of <p » Where a and b are some minimum 
value of the characteristics, then we can say that the effectiveness 


will be zero. We may be able to establish two other constants p, and 


(3) (3) 
2 


q such that if; Cy p andc q the effectiveness will be 


complete or 100%, These constants establish the end points of a 


ratio scale and we require the effectiveness as a function of te 


(3) 
2 


and c when: 


a ay ee and b <= ae <q =. 


From this point we can assign the problem to a decision maker's 


utility function or apply some model such as eq. 2. 
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